Running head: APPLYING KNOWLEDGE OF QUANTITATIVE 



Applying Knowledge of Quantitative Design and Analysis 
Richard S. Baskas 
Walden University 
August 28, 2011 




APPLYING KNOWLEDGE OF QUANTITATIVE 



2 



Abstract 

This study compared and contrasted two quantitative scholarly articles in relation to their 
research designs. Their designs were analyzed by the comparison of research references and 
research specific vocabulary to describe how various research methods were used. When 
researching and analyzing quantitative scholarly articles, it is imperative to understand how 
research designs can be the same, but their methods of analysis can differ depending on the 
study, and how vocabulary describes the research being conducted. The course instructor 
provided two quantitative articles which were analyzed by research references provided by the 
student. A literature review of the two articles and that of the research references were 
conducted to provide a more in-depth explanation of the research designs. Analysis revealed that 
even though both articles shared the same research design, various analytical techniques that are 
used can produce different perspectives of the results. 
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Applying Knowledge of Quantitative Design and Analysis 
When examining interrelationships among variables, where objective theories are tested, 
the most logical method to use would be a quantitative design. These variables can be measured, 
typically by using instruments of predetermined, closed-ended questions, so that numbered data 
can be analyzed using statistical procedures (Creswell, 2009; Trochim & Donnelly, 2008). 
According to Gall, Gall, and Borg (2003) (as cited by Wiersma & Jurs, 2009, p. 118), 
quantitative research has its roots in positivist philosophy, “the epistemological doctrine that 
physical and social reality is independent of those who observe it, and that observations of this 
reality, if unbiased, constitute scientific knowledge” (p. 632). Even though the research 
provided in quantitative articles may begin with the same design methods, various design 
characteristics and analytical techniques can provide for a diverse analysis. 

Research Design and Rationale 

When deciding to use a quantitative research design, a rationale should follow to explain 
the purpose of the design. The research an author intends to conduct will often dictate the type 
of design method (quantitative, qualitative, or mixed methods) that will be used to produce 
results. Analysis of two articles, Anderson, Murray, and Olivarez (2002), and Backhaus and Liff 
(2007), revealed a lack of terminology that described their design method and a lack of an 
explanation of the study. The authors of both articles provided numerous research questions but 
no hypotheses which suggested an experimental design, “a hypothesis that is tested to establish a 
cause-and-effect relationship” (Lodico, Spaulding, & Voegtle, 2010, p. 12). 
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Results 

Method of Data Collection 

Researchers of both articles distributed surveys to collect data. Anderson, Murray, and 
Olivarez (2002) did not mention the respondents who completed their surveys. Backhaus and 
Liff (2007) allowed students plenty of time to complete their surveys during class time. Data of 
both articles were tallied and recorded into tables. Anderson, Murray, and Olivarez (2002) were 
quite detailed in their study as they used a number of tables to arrange and display the 
managerial roles and regions of span of control. Backhaus and Liff (2007) condensed their data 
into a few tables as they compared a number of cognitive styles with that of gender roles. 

Data Analysis 

Descriptive statistics. Backhaus and Liff (2007) developed and analyzed ten hypotheses 
by comparing cognitive styles to student grade point averages (GPA), students’ gender, means 
(arithmetic averages), and standard deviations. They displayed their findings in a few tables with 
both surveys and cognitive styles on the left and means, standard deviations, and relationships 
between cognitive styles listed at the top of table. 

Anderson, Murray, and Olivarez (2002) analyzed their data by using numerous tables that 
included number of responses, means, and standard deviations. These tables listed different 
types of data to display relationships between (1) managerial roles and influencing variables, (2) 
how managers perform managerial roles in the United States, and (3) span of control by regions 
(North Central, Middle States, New England, Northwest, Southern, and Western) that contained 
percentages of gender, age, years of managerial experience, years at current institution, and years 



at their current position. 
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Causal-comparative statistics. Anderson, Murray, and Olivarez (2002) found 
significant relationships with their independent variables (environment, personal, and situational) 
and dependent variables (10 managerial roles). Multivariate analysis of variance (MANOVA) 
proved significant differences between the independent and dependent variables where univariate 
tests confirmed the findings. Analysis of variance (ANOVA) proved differences in roles that 
were related to gender. 

Backhaus and Liff (2007) found CSI, RASI scales and other variables to be related. The 
internal reliability for CSI was acceptable with a Cronbach’s alpha coefficient of .79. Internal 
reliability for RASI ranged from .83 (for the Strategic Approach Scale) to .64 (for the 
Metacognitive Scale). This study found significant relationships between (1) CSI and GPA, (2) 
gender and cognitive styles, (3) various approaches to studying, (4) GPA and approaches to 
studying, and (5) mean scores on CSI. 

Initial Critique 

Research Questions or Hypotheses 

Both articles lacked a hypothesis but listed a number of research questions that could be 
answered through the “systematic collection of data and clearly meet ethical guidelines” (Lodico, 
Spaulding, & Voegtle, 2010, p. 27) to guide their research. Anderson, Murray, and Olivarez 
(2002) (as cited by Anderson, Murray, & Olivarez, 2002, p. 4) listed their research questions (1) 
What managerial roles do community college Chief Academic Officers perform and which roles 
do they emphasize? (2) Are there differences in their roles due to environmental characteristics, 
personal characteristics, or situational characteristics? Backhaus and Liff (2007) listed their 
questions (1) How do students process information? (2) Can they grasp the big picture? (3) Are 
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they studying the appropriate materials in the right ways to be prepared as managers and leaders 
(pp. 445-446)? 

Definitions 

Only a few definitions were used between both articles. According to Murray, Murray, 
and Summar (2000) (as cited by Anderson, Murray, & Olivarez, 2002, p. 1), a Chief Academic 
Officer (CAO) is an “administrative head of academic programs with responsibility for all 
academic affairs at the institution”. A manager is a “leader of a particular organizational unit” 
(Anderson, Murray, & Olivarez, 2002, p. 3). According to Messick (1984), and Riding and 
Rayner (1997) (as cited by Backhaus & Liff, 2007, p. 446), cognitive style is a “consistent 
preference for gathering and understanding information”. 

Measurement and Instrumentation 

Both articles used a very minute measurement scale for their instrument. Anderson, 
Murray, and Olivarez (2002) utilized a four-part scale (3) not used at all , (6) a little , (9) some , 
and (12) often. Backhaus and Liff (2007) used a trichotomous scale (1) true , (2) false, and (3) 
uncertain for the CSI and a 5-point Likert-type scale for the RASI survey. 

For their instrumentation, Anderson, Murray, and Olivarez (2002) used a managerial role 
survey (Mintzberg, 1973) which was designed by Judson (1981) (as cited by Anderson, Murray, 
& Olivarez, 2002). Backhaus and Liff (2007) used a questionnaire that was comprised of two 
parts (1) CSI (Allinson & Hayes, 1996) (as cited by Backhaus & Liff, 2007), composed of 38 
items that measured cognitive styles; and (2) RASI (Entwisitle & Tait, 1994) (as cited by 
Backhaus & Liff, 2007), composed of 44 items. 
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Validity and Reliability Processes 

Changes were made for both instruments to redefine their purposes. The managerial role 
survey (Mintzberg, 1973) (as cited by Anderson, Murray, & Olivarez, 2002) that Anderson, 
Murray, & Olivarez (2002) used was redefined further by Mech (1997) (as cited by Anderson, 
Murray, & Olivarez, 2002) and later again by researchers for this study. Ten more questions 
were added due to latent factors having fewer than three indicator variables. The survey was 
further critiqued by ten high education managers for content validity of managerial roles (latent 
factors). These managers were asked to identify which combination of three questions 
corresponded to each of the ten managerial roles, but none of the educators agreed. Wording 
revisions were made for three questions. The survey was further analyzed for internal 
consistencies. Further validity included an exploratory factor analysis that all 30 questions 
loaded onto nine independent factors. The survey’s reliability was not stated. 

The CSI questionnaire (Backhaus & Liff, 2007) was initially validated by Allinson and 
Hayes (1996) (as cited by Backhaus & Liff, 2007) and later used in studies of management 
students and managers (e.g., Allinson, Armstrong, & Hayes, 2001; Sadler-Smith, 1999; Sadler- 
Smith, Allinson, & Hayes, 2000) (as cited by Backhause & Liff, 2007). Duff (2000) (as cited by 
Anderson, Murray, & Olivarez, 2002) had five items, measuring collaborative approaches to 
studying, added to RASI. Duff (2000) (as cited by Anderson, Murray, & Olivarez, 2002) 
reported the reliability of previous RASI studies showing alpha coefficients in the .70+ range. 
The Collaborative Scale consisted of five items related to desire to work together with other 
students. Sadler-Smith (1999) (as cited by Anderson, Murray, & Olivarez, 2002) reported this 
scale yielded strong internal reliability of .84. 
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Sampling Procedures 

The process of choosing samples was different for both studies. Anderson, Murray, and 
Olivarez (2002) used a stratified random sample for their study by six national accrediting 
regions, then calculating percentages of community colleges in each region. An appropriate 
number of colleges from each accreditation region were randomly selected. For statistical 
analysis purposes, oversampling was necessary. Surveys with instructions and a self-addressed, 
stamped envelope were mailed to randomly selected CAOs. Two weeks later nonrespondents 
received a second mailing. The final population used was 180 respondents. The sample type 
used in Backhaus and Liff s study (2007) was not mentioned but included undergraduate 
business students (management, accounting, statistics, and marketing) who were attending a 
comprehensive college in the Northeast. The final population used was 222 respondents. 

Ethical Considerations 

Protection from harm. It was evident that the authors showed respect for their 
participants as it was never mentioned. Backhaus and Liff (2007) administered their survey to 
students in class and allowed them to complete during class time. Anderson, Murray, and 
Olivarez (2002) mailed their surveys, with instructions and a self-addressed, stamped envelope, 
to randomly selected CAOs. 

Informed consent. In their study (Backhaus & Liff, 2007) of how to improve students’ 
learning, it would be obvious that their students would not need to be informed of the purpose of 
the study as this most likely would change their students’ view of the study. Anderson, Murray, 
and Olivarez (2002) did not mention if their participants were contacted before their study began. 
It would seem that it would be more time consuming calling or contacting them by mail 



beforehand. 




APPLYING KNOWLEDGE OF QUANTITATIVE 



9 



Confidentiality. The class instructor’s name and behavior (Backhaus & Liff, 2007) were 
not mentioned so it can be assumed that permission was granted to allow these students to 
participate in the study. Participant names were also not used (Anderson, Murray, & Olivarez, 
2002) as surveys were sent randomly in hopes to obtain as many responses as possible. 

Conclusion 

Even though both articles began with a quantitative method, no two articles are the same 
as their research can proceed in different directions based on their study. Both articles shared the 
experimental design that lead to the development of a survey. An initial critique revealed that 
research questions were used in both articles, but no hypotheses. Definitions explained various 
terminologies. Preestablished surveys were considered and later modified to ensure their 
validity. Survey measurements consisted of three or four parts for the scale. Samples used were 
either stratified random or not mentioned. Both articles showed respect towards their 
participants. No harm was done as surveys were completed with ease. Students’ behaviors were 
not influenced before the study was conducted as well as their names were not mentioned to 



protect their identities. 
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